1. Technical Field
The invention is related to computer operating systems and in particular to a computer operating system which is highly componentized and has dynamically loadable operating features which may be loaded and unloaded during system run time.
2. Background Art
The progressive computerization of society involves a number of diverse computing platforms beside the general-purpose computer:
Embedded control systems, including consumer devices, intelligent sensors and smart home controls.
Communication-oriented devices such as digital cell phones and networking infrastructure.
Programmable peripherals and microcontrollers.
In all these cases, the general-purpose platform approach is either not applicable, or it is prohibitively expensive. The microprocessor might be a DSP, a VLIW, or a micro-controller; the memory budget is severely restricted; there might be no MMU; the network connection might be sporadic; and Real-Time support is essential.
Current operating systems are either inflexible, big, lack Real-Time support, have complex hardware requirements, or are so special purpose that good development tools are unavailable and code reusability is low.
Microkernels [Black92, Engler95] attempt to modularize the operating system. But they confuse modularity with security by mandating that system services be in separate address spaces. Many of the services moved into separate server processes are still necessary for these systems to function and often the services have to trust each other.
C++ and Java provide objects at a very fine granularity level, and they are extremely successful with application programmers. Unfortunately, both languages confine their objects to a single address space. Object Linking and Embedding (OLE) [Brockschmidt95] and other similar systems extend objects across address spaces and across machine boundaries. OLE seamlessly integrates independently developed components. When editing an Excel spreadsheet inside a Word document it is in fact the Excel process that operates on objects inside of Word""s address space. Unfortunately, it only works for user mode applications.
Modularity has always been an important paradigm in software design. By breaking a complex system into pieces, the complexity becomes more manageable. Address spaces provide security by installing virtual-memory based firewalls between applications. These two issues are orthogonal, but the distinction has been lost in systems research that has been concentrating on so-called microkernels. These issues have been discussed in the following publications:
[Bershad95] Brian Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. Fiuczynski, D. Becker, S. Eggers, C. Chambers. Extensibility, safety and performance in the Spin operating system. In 15th ACM Symposium on Operating System Principles, pages 267-284, Copper Mountain Resort, Colo., December 1995.
[Black92] David Black, David Golub, Daniel Julin, Richard Rashid, Richard Draves, Randall Dean, Alessandro Forin, Joseph Barrera, Hideyuki Tokuda, Gerald Malan, David Bohman. Microkernel Operating System Architecture and Mach. In 1st USENIX Workshop on Micro-kernels and Other Kernel Architectures, pages 11-30, Seattle, April 1992.
[Brockschmidt95] K. Brockshmidt. Inside OLE, Second ed. Microsoft Press, Redmond Wash., 1995.
[Cheriton94] David Cheriton, Kenneth Duda. A Caching Model of Operating System Kernel Functionality. In 1st Symposium on Operating Systems Design and Implementation, Seattle, 1994.
[Cheriton88] David Cheriton. The V distributed system. In Communications of the ACM, pages 314-333, March 1988.
[Draves97] Richard Draves, Scott Cutshall. Unifying the User and Kernel Environments. Microsoft Research Technical Report MSR-TR-97-10, 16 pages, March 1997
[Engler95] D. R. Engler, M. F. Kaashoek, J. O"" Toole Jr. Exokernel: an operating system architecture for application-specific resource management. In 15th ACM Symposium on Operating System Principles, pages 251-266, Copper Mountain Resort, Colo., December 1995.
[Ford97] Bryan Ford, Godmar Back, Greg Benson, Jay Lepreau, Albert Lin, Olin Shivers. The Flux OSKit: A Substrate for Kernel and Language Research. In Proceedings of the 16th ACM Symposium on Operating Systems Principles, pages 38-51. ACM SIGOPS, Saint-Malo, France, October 1997.
[Golub90] David Golub, Randall Dean, Alessandro Forin, Richard Rashid. UNIX as an application program. In USENIX 1990 Summer Conference, pages 87-95, June 1990.
[Helander94] Johannes Helander. Unix under Mach: The Lites Server. Master""s thesis, 71 pages, Helsinki University of Technology, 1994. Available from http://www.cs.hut.fi/xcx9cjvh/lites.MASTERS.ps
[Hildebrand92] D. Hildebrand. An architectural overview of QNX. In 1st USENIX Workshop on Micro-kernels and Other Kernel Architectures, pages 113-126, Seattle, April 1992.
[ISI95] Integrated Systems Inc. pSOSystem System Concepts. Part No. COL0011, May 1995, ISI, Sunnyvale Calif.
[Jones96] Michael B. Jones, Joseph S. Barrera, III, Richard P. Draves, Alessandro Forin, Paul J. Leach, Gilad Odinak. An Overview of the Rialto Real Time Architecture. In Proceedings of the 7th ACM SIGOPS European Workshop, pagg. 249-256, September 1996.
[Jones97] Michael B. Jones et al. CPU Reservations and Time Constraints: Efficient, Predictable Scheduling of Independent Activities. In Proceedings of the 16th ACM Symposium on Operating Systems Principles, pages 198-211. ACM SIGOPS, Saint-Malo, France, October 1997.
[Jones 97b] Michael B. Jones. The Microsoft Interactive TV System: An Experience Report. Microsoft Research Technical Report MSR-TR-97-18, July, 1997.
[Julin9l] Daniel Julin, Jonathan Chew, Mark Stevenson, Paulo Guedes, Paul Neves, Paul Roy. Generalized Emulation Services for Mach 3.0: Overview, Experiences and Current Status. In Proceedings of the Usenix Mach Symposium, 1991.
[Lee98] Dennis Lee, Patrick Crowley, Jean-Loup Baer, Tom Anderson, Brian Bershad. Execution characteristics of desktop applications on Windows NT. In Proceedings of the 25th International Symposium on Computer Architecture, Barcelona, Spain, June 1998.
[Liedtke95] Jochen Liedtke. On xe2x96xa1-kernel construction. In 15th ACM Symposium on Operating System Principles, pages 237-250, Copper Mountain Resort, Colorado, December 1995.
[Mogul87] Jeffrey Mogul, Richard Rashid, Michael Accetta. The Packet Filter: an Efficient Mechanism for User-level Network Code. In 11th ACM Symposium on Operating System Principles, November 1987.
[Rashid87] Richard Rashid. From RIG to Accent to Mach: The evolution of a network operating system. Carnegie Mellon University Technical Report, August 1987.
[Rozier88] M. Rozier, A. Abrassimov, F. Armand, I. Boule, M. Gien, M. Guillemont, F. Hermann, C. Kaiser, S. Langlois, P. Leonard, W. Neuhauser. CHORUS distributed operating system. In Computing Systems, pages 305-370, Vol. 1-4, 1988.
[Young89] Michael Wayne Young. Exporting a User Interface to Memory Management from a Communication-Oriented Operating System. Ph.D. Thesis CMU-CS-89-202, Carnegie Mellon University, November 1989.
Mach [Black92] defined an interface for external memory managers [Young89] and was able to split virtual memory into functionally distinct parts, allowing part of the functionality to reside outside the privilege-level component (the xe2x80x9ckernelxe2x80x9d). Mach also separated part of the Unix operating system services out of the kernel [Golub90, Helander94], achieving modularity but limited additional functionality. The multiserver project [Julin91] went further in the modularization by splitting the Unix services into multiple independent servers. The componentization added structure and generality to the services. However, keeping the services in multiple address spaces did not add any security or robustness since components had to be available and trusted in any case. The most interesting new functionality was in the ability to emulate multiple OS interfaces, at the same time.
Contemporary research systems take the minimization of the xe2x80x9ckernelxe2x80x9d concept even further by defining even lower level abstractions and demonstrating the ability to split states across address space boundaries. None of these systems defines a new application programming interface (API) different from the Unix they emulate. The API that their predecessors [Rashid87, Cheriton88, Rozier88] did define, based on RPC and message exchanges, were not very successful with programmers.
The Cache Kernel [Cheriton94] uses Mach""s external memory manager metaphor uniformly for the management of all kernel objects. Threads, Address Spaces and User Kernels are all handled through this page in-pageout logical interface. An actual application is statically linked with a number of libraries, which provide default implementations of the required User Kernel components (VM, scheduling, IPC). This offers some flexibility by letting untrusted applications have their custom application kernel. Overall complexity is not decreased; it seems an application kernel would have to be as complicated as any other operating system. The ability to write your own application kernel would seem useful for a limited number of users, in teaching operating systems for instance.
Exokernel [Engler95] goes along the same lines demonstrating further ability to run operating system code in user mode. While it is highly successful in this and offers some added flexibility, it is questionable whether the premises differ from that of microkernels. The main contribution is in the mechanisms for application-specific resource management.
[Liedtke95] argues that microkernels have failed exclusively on performance grounds, and that poor performance is their only cause for inflexibility. Our argument is the opposite: inflexibility is inherent in the design, and leads to unavoidable inefficiencies that can only be mitigated by good implementations, never eliminated.
Spin [Bershad95] addresses the issue of expensive address space crossings by letting user code compiled by a trusted compiler run inside the kernel. This can be viewed as smart proxies that can do a lot of the work locally that otherwise would require communication. It is similar to loading packet filters into network drivers [Mogul87], to running database application query language inside database engines [reference], or to sandboxing Java applets. Applying these techniques to operating systems is beneficial when a trust boundary must be crossed and the cost would otherwise be high. It does not address the issue of whether or not a trust boundary is necessary. Spin uses an object-based language (Modula3) to provide extensibility. The pointer-safety property of the language is what permits execution of untrusted code in privileged mode. Trust relationships, as in the user-versus-kernel separation, should not dominate system decomposition. It is important to return to a global system view. The present invention addresses the issue of how to minimize the set of base services, and how to dynamically extend them on demand.
[Ford97] shows how a base set of system components can be composed in different ways to build an operating system kernel. The granularity is fairly coarse, and the techniques are limited to static linking. Components that should be of interest to OS researchers (VM, IPC, scheduling, etc.) cannot be replaced or removed, neither statically nor dynamically. The decomposition is otherwise limited to the xe2x80x9cOSxe2x80x9d component; it is not meant as a whole-system approach. This does not go far enough in the componentization. It provides a few convenient components, such as bootstrap loader and filesystems, but is mostly concerned with reusing existing device drivers and Unix code. It fails to componentize the core kernel services or extend the paradigm to applications.
Componentization and location independence has also been studied in the context of filesystems and network protocols [Maeda93] and in a number of existing embedded systems, such as pSOS [ISI95]. In a typical embedded system there is no loader, and components can only be chosen at static link time when the load image is built. Services are extremely limited, sometimes exclusively to the scheduling component. The number and priority of threads might have to be specified statically as well.
Chorus [Rozier88] can be configured to use either a page-based or a segment-based VM system.
A preferred embodiment of the invention is directed to a flexible system architecture that is suitable for a wide range of applications. The system is built out of minimal but flexible components, which can be deployed as needed. Instead of mandating a fixed set of operating system services and hardware requirements, the system preferably provides a menu of well-defined components that can be chosen to compose a complete system depending on hardware capabilities, security needs, and application requirements.
Dynamic loading and unloading of components provides the flexibility that lets the system adapt to changing requirements.
The componentization makes it possible to change the implementation of a component without affecting the rest of the system. Minimalism makes it possible to use the system with severely restricted hardware budgets. It also forces the system to be understandable and flexible. Software components, when possible, are not tied to a particular layer of the system, but can be reused. For example, the same code that implements the system physical memory heap is used to provide application heaps over virtual memory. The key system building blocks are componentized. This includes the virtual memory system, IPC, and the scheduler in addition to filesystems, networking, drivers, and protection policies. Preferred embodiments of the present invention extend object-orientation both across address spaces and across protection levels.
In a preferred embodiment, components are located in separate address spaces only when there is a real reason for it, such as security or specific address space requirements. Thus, the price of multiple address spaces (and transitions thereof) is paid only where needed.
The invention is directed toward a loadable interprocess communication manager and generally to a computer operating system capable of supporting plural threads running in a computer having a working memory, the computer operating system including a kernel resident in the working memory at link time and a loadable interprocess communication manager resident at link time outside of the working memory and dynamically loadable into the working memory at run time upon request by one of the threads in one address space to communicate with an other thread in an other address space. The kernel includes a loader for loading the interprocess communication manager into the working memory in response to the request by the one thread. The computer further includes a storage memory separate from the working memory, the loadable interprocess communication manager residing at link time in the storage memory. The loader loads the interprocess communication manager from the storage memory to the working memory. The loadable interprocess communication manager is terminable from the working memory upon lack of a need for communication between threads in different address spaces.
Preferably, the kernel of the operating system includes an interprocess communication fault handler for interprocess communication faults occurring in the absence of the interprocess communication manager in the working memory.
The kernel also includes a Namespace for registering the interprocess communication manager upon the interprocess communication manager being loaded into the working memory, whereby the interprocess communication manager becomes available to each thread through the Namespace. The Namespace includes an object supporting plural interfaces exposable by the threads, the plural interfaces including a query interface through which a thread invokes the interprocess communication manager, an add reference interface by which the Namespace manages each request for the interprocess communication manager from any of the threads, and a release reference interface, by which Namespace manages termination of the interprocess communication manager from the working memory. The loader is responsive to the Namespace in maintaining the interprocess communication manager in the working memory whenever the add reference interface has a reference to the interprocess communication manager.
The loadable interprocess communication manager includes an object with plural interfaces exposable to other objects, the plural interfaces of the interprocess communication manager supporting methods including an interprocess communication trap handler (IPC Trap Handler) and copy (IPC Copy).
The operating system can include plural loadable interprocess communication managers resident outside of the working memory at link time, the loader being capable of loading different ones of the plural interprocess communication managers into the working memory at run time.